Skip to content

Suppress benign camembert/roberta model type warning in ThEnTranslator - #1215

Merged
bact merged 8 commits into
devfrom
copilot/fix-camembert-roberta-issue
Feb 3, 2026
Merged

Suppress benign camembert/roberta model type warning in ThEnTranslator#1215
bact merged 8 commits into
devfrom
copilot/fix-camembert-roberta-issue

Conversation

Copilot AI commented Jan 19, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

What do these changes do

Suppresses the transformers warning "Using a model of type camembert to instantiate a model of type roberta" when loading the scb_1m_th-en_spm translation model in ThEnTranslator.__init__().

What was wrong

The pre-trained scb_1m_th-en_spm model checkpoint contains a CamemBERT configuration but fairseq loads it as a RoBERTa-compatible transformer. This triggers a model type mismatch warning from the transformers library during test execution.

How this fixes it

Wraps TransformerModel.from_pretrained() in a warnings.catch_warnings() context manager with a targeted filter for the specific warning pattern. The filter uses case-insensitive regex matching to handle potential message variations.

with warnings.catch_warnings():
    warnings.filterwarnings(
        "ignore",
        message="(?i).*using a model of type .* to instantiate a model of type.*",
    )
    self._model = TransformerModel.from_pretrained(...)

Your checklist for this pull request

  • Passed code styles and structures
  • Passed code linting checks and unit test
Original prompt

This section details on the original issue you should resolve

<issue_title>bug: translate: Using camembert model to instantiate roberta model</issue_title>
<issue_description>### Description

"You are using a model of type camembert to instantiate a model of type roberta. This is not supported for all configurations of models and can yield errors." warning issued during test_translate test.

Possibly from scb_1m_th-en_spm model.

Expected results

No warning

Current results

See https://github.com/PyThaiNLP/pythainlp/actions/runs/7861725110/job/21450401313#step:5:1449

Steps to reproduce

Run test_translate.py

PyThaiNLP version

5.0.1

Python version

3.8

Operating system and version

Linux and Mac

More info

No response

Possible solution

No response

Files

Relevant code blocks:

pythainlp/tests/test_translate.py

Lines 12 to 15 in c8b547e

class TestTranslatePackage(unittest.TestCase):
def test_translate(self):
# remove("scb_1m_th-en_spm")
self.assertIsNone(download_model_all())

pythainlp/pythainlp/translate/en_th.py

Lines 37 to 42 in c8b547e

def download_model_all() -> None:
"""
Download all translation models in advance
"""
_download_install(_EN_TH_MODEL_NAME)
_download_install(_TH_EN_MODEL_NAME)

pythainlp/pythainlp/translate/en_th.py

Lines 19 to 25 in c8b547e

_EN_TH_MODEL_NAME = "scb_1m_en-th_moses"
# SCB_1M-MT_OPUS+TBASE_en-th_moses-spm_130000-16000_v1.0.tar.gz
_EN_TH_FILE_NAME = "SCB_1M-MT_OPUS+TBASE_en-th_moses-spm_130000-16000_v1.0"
_TH_EN_MODEL_NAME = "scb_1m_th-en_spm"
# SCB_1M-MT_OPUS+TBASE_th-en_spm-spm_32000-joined_v1.0.tar.gz
_TH_EN_FILE_NAME = "SCB_1M-MT_OPUS+TBASE_th-en_spm-spm_32000-joined_v1.0"
</issue_description>

Comments on the Issue (you are @copilot in this section)

@wannaphong I have a computing problems , so I will change this issues as future plan.

Oh wrong issue</comment_new>

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits January 19, 2026 14:54
@wannaphong
Add warning filter to suppress camembert/roberta model type warning
224b9b8 
Co-authored-by: wannaphong <8536487+wannaphong@users.noreply.github.com>
@wannaphong
Make warning filter pattern case-insensitive
5a63099 
Co-authored-by: wannaphong <8536487+wannaphong@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix bug using camembert model to instantiate roberta model Suppress benign camembert/roberta model type warning in ThEnTranslator Jan 19, 2026
Copilot AI requested a review from wannaphong January 19, 2026 14:57
@bact

bact commented Jan 19, 2026

Copy link
Copy Markdown
Member

@copilot There are changes in dev branch to fix the errors occured in Python 3.13. Merge the changes and run the tests again.

Copilot AI commented Jan 19, 2026

Copy link
Copy Markdown
Contributor Author

@copilot There are changes in dev branch to fix the errors occured in Python 3.13. Merge the changes and run the tests again.

Merged dev branch successfully in commit 3c56c3f. The Python 3.13 syntax fixes are now included, and all linting checks pass.

Copilot AI requested a review from bact January 20, 2026 00:00
@coveralls

coveralls commented Jan 20, 2026 •
edited
Loading

Copy link
Copy Markdown

Coverage Status

coverage: 65.799% (-0.02%) from 65.821%
when pulling 1f7959c on copilot/fix-camembert-roberta-issue
into 8c880d9 on dev.

@bact
bact marked this pull request as ready for review January 20, 2026 08:00
@bact

bact commented Feb 1, 2026

Copy link
Copy Markdown
Member

@copilot The dev branch is updated. Some code were refactored. Please resync with it.

The test suite got refactored and now we have 4-tier of test cases located in four test suites: core, compact, extra, noauto - depends on their dependency requirements.
Some test cases were reorganized and moved to other test suites.

See the test suite refactoring and naming conventions in this merged PR
#1248
and follow the new convention.

Copilot AI commented Feb 1, 2026

Copy link
Copy Markdown
Contributor Author

@copilot The dev branch is updated. Some code were refactored. Please resync with it.

The test suite got refactored and now we have 4-tier of test cases located in four test suites: core, compact, ...

Resynced with dev branch in commit ee75831. The branch now includes the 4-tier test suite refactoring (core, compact, extra, noauto) with updated naming conventions. The translate tests are already using the correct TranslateTestCaseX class name in tests/extra/testx_translate.py, and all code passes linting checks.

@bact

bact commented Feb 1, 2026

Copy link
Copy Markdown
Member

@copilot The dev branch has updates. Sync changes.

Copilot AI commented Feb 1, 2026

Copy link
Copy Markdown
Contributor Author

@copilot The dev branch has updates. Sync changes.

Synced with latest dev branch in commit 80fa60e. The branch now includes the latest documentation updates, contribution guidelines, and workflow optimizations. All code passes linting checks.

@bact

bact commented Feb 2, 2026

Copy link
Copy Markdown
Member

@copilot The dev branch has updates. Sync changes.

Copilot AI commented Feb 2, 2026

Copy link
Copy Markdown
Contributor Author

@copilot The dev branch has updates. Sync changes.

Synced with latest dev branch in commit 1f7959c. The branch now includes significant test coverage improvements, enhanced NER and tokenization tests, test helper utilities, and release automation improvements. All code passes linting checks.

@sonarqubecloud

sonarqubecloud Bot commented Feb 2, 2026

Copy link
Copy Markdown

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@bact
bact requested a review from Copilot February 2, 2026 10:47
Copilot AI reviewed Feb 2, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR suppresses a known, benign transformers warning when loading the scb_1m_th-en_spm model in ThEnTranslator, and updates project citation metadata with a DOI. The goal is to keep test output clean while preserving existing translation behavior and to improve citation information.

Changes:

  • Wrap TransformerModel.from_pretrained in ThEnTranslator.__init__ with a warnings.catch_warnings() context and a targeted, case-insensitive filterwarnings regex for the specific model-type mismatch warning.
  • Add import warnings to pythainlp/translate/en_th.py to support the new suppression logic.
  • Add the Zenodo DOI (10.5281/zenodo.3519354) to CITATION.cff for more complete citation metadata.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
pythainlp/translate/en_th.py Introduces a scoped warnings filter around TransformerModel.from_pretrained to ignore the specific camembert→roberta model-type warning while loading the Thai–English translation model.
CITATION.cff Adds the Zenodo DOI entry to align citation metadata with the published record.

@bact

bact commented Feb 2, 2026

Copy link
Copy Markdown
Member

@wannaphong please see if this one is ready.

@bact bact added this to the 5.3 milestone Feb 3, 2026
@bact
bact merged commit 0eaff7f into dev Feb 3, 2026
29 of 39 checks passed
@bact
bact deleted the copilot/fix-camembert-roberta-issue branch February 3, 2026 20:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments
@bact bact bact approved these changes
+1 more reviewer
@wannaphong wannaphong wannaphong approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@wannaphong wannaphong

Copilot code review Copilot

Labels

None yet

Projects

None yet

Milestone

5.3

Development

Successfully merging this pull request may close these issues.

bug: translate: Using camembert model to instantiate roberta model

5 participants

@bact @coveralls @wannaphong